Learning Bayesian Networks Using Feature Selection

This paper introduces a novel enhancement for learning Bayesian networks with a bias for small, high-predictive-accuracy networks. The new approach selects a subset of features that maximizes predictive accuracy prior to the network learning phase. We examine explicitly the eeects of two aspects of the algorithm, feature selection and node ordering. Our approach generates networks that are computationally simpler to evaluate and display predictive accuracy comparable to that of Bayesian networks which model all attributes. 28.1 Introduction Bayesian networks are being increasingly recognized as an important representation for probabilistic reasoning. For many domains, the need to specify the probability distributions for a Bayesian network is considerable, and learning these probabilities from data using an algorithm like K2 Cooper92] could alleviate such speciication diiculties. We describe an extension to the Bayesian network learning approaches introduced in K2. Our goal is to construct networks that are simpler to evaluate but still have high predictive accuracy relative to networks that model all features. Rather than use all database features (or attributes) for constructing the network, we select a subset of features that maximize the predictive accuracy of the network. Then the learning process uses only the selected features as nodes in learning the Bayesian network. We examine explicitly the eeects of two aspects of the algorithm: (a) feature selection, and (b) node ordering. Our experimental results verify that this approach generates networks that are compu-tationally simpler to evaluate and display predictive accuracy comparable to the predictive accuracy of Bayesian networks that model all features. Our results, similar to those observed by other studies of feature selection in learning Caruana94, John94, Langley94a, Langley94b], demonstrate that feature selection provides comparable predictive accuracy using smaller networks. For example, by selecting as few as 18% of the features for the